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ABSTRACT 



The report describes the Instructional Program Review 
Process at Austin Community College (Texas) . The report includes program 
review process timeline and schedule. The Instructional Program Review 
Process focuses on issues of need, cost, and effectiveness of instructional 
programs. It provides for identification of improvements, includes standards 
of academic quality, and allows for better decision-making, including 
modifications, budget, strategic planning. In consultation with the 
administration, Deans determine the order in which programs in their areas 
will be reviewed. Each dean will designate a self-study team for each 
instructional area. The selection will be based on recommendations from the 
Program Coordinator or Taskforce Chair. The Chair is the key member of the 
self-study team. The Chair's responsibility begins during the spring term 
prior to the self-study and continues through the next academic year until 
the final self-study report has been submitted. The basis of the program 
review process is the analysis of information relating to a set of core 
indicators. These core indicators have been identified by the Program Review 
Work Group and reflect the input from a variety of sources. They include 
indicators federal, state, and regional agencies, as well as "best practices" 
identified in similar processes at other community colleges. The core 
indicators are presented in three sets, categorized by need, cost, and 
program effectiveness. (JA) 
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Instructional Program Review Process 



Overview 

The Instructional Program Review Process focuses on aspects of need, cost and 
effectiveness of instructional programs. Specifically, is the instructional quality of the 
program high enough to meet institutionally set standards? 

What is a program? 

For purposes of program review, a program is an organizational unit within the 
college that provides instructional or support services. These include instructional 
disciplines, degree and credentialing, and non-instructional services, including non- 
classroom instructional services. The process outlined here focuses on instructional 
areas. 

Purpose of Program Review Process 

• Continuous improvement of programs 

• Assessment of strengths and weaknesses of a program as well as the opportunities 
and threats it faces. 

• Determination of the capabilities of the program/area 

• Break down barriers between areas of operation 

• Ensure best uses of resources — fiscal, human, facilities, equipment, technology 

• Articulate program’s mission and vision 

• Provide the foundation for application to the Greater Austin Quality Council 
(Baldrige Criteria) 

• Responds to Board policy 

Characteristics of Program Review Process 

The Instructional Program Review Process at Austin Community College: 

• Provides for identification of improvements 

• Includes standards of academic quality 

• Allows for better decision-making, including modifications, budget, strategic 
planning 

• Is integrated into the Institutional Effectiveness process 

• Is integrated into other external review process, including the THECB site review, 
SACS accreditation, program-specific reviews required by other agencies 

• Ensures customer focus integrated into program 

• Creates an opportunity for broad-based input from constituencies including students, 
staff, faculty, employers 
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• Is a cyclical process with the expectation that there will be continuous review within 
the program 

• Assesses the degree to which the program is fulfilling its mission and accomplishing 
its goals 



Administration of the Instructional Program Review Process 

Oversight of the Program review process in the instructional areas is the 
responsibility of the Instructional Program Review Committee. Membership includes 
one faculty member from each dean area. Appointments are made biennially and 
members serve for 2 years. The Associate Vice President for Institutional Effectiveness 
serves as an ad hoc member. This Committee is chaired by a member of the Committee 
selected by the Executive Vice President for Instruction. 

The responsibilities of the Instructional Program Review Committee include: 

• Overseeing the Program Review Process to ensure that it occurs within the 
stated timeline 

• Monitoring of the implementation of the Program Review Process 

• Providing assistance to units involved in the process 

• Evaluating the overall process 

• Developing modifications as necessary to ensure the process functions 
effectively 

• Developing modifications of the Indicators of Effectiveness, including related 
criteria, as needed 

• Reviewing self-study reports 



Guidelines for Implementation 

In consultation with the administration. Deans determine the order in which 
programs in their areas will be reviewed 

New programs will be incorporated into the timeline as they are developed. In 
most cases, program review will not occur during the first 3 years of program 
implementation. 

The Office of Institutional Effectiveness will provide data related to the program 
review. Specific types of information are indicated below. 

Self Study Team 

Each Dean will designate a Self-study Team for each instructional area. The 
selection will be based on recommendations from the Program Coordinator or Taskforce 



Chair. The primary responsibilities of the Self-study Team are to collect and analyze the 
information, develop the Self-study Report, and make recommendations. 

It is anticipated that the Team will meet on a regular basis during the fall 
semester. These meetings may be augmented with discussions via email or telephone. 

The Chair is the key member of the Self-study Team. The Chair’s responsibility 
begins during the Spring Term prior to the Self-study and continues through the next 
academic year until the final Self-study report has been submitted. 

The membership of the Self-study Team will include: 



Member 


# 


Role 


Chair 


1 


-Coordinates Self-study 

-Coordinates departmental surveys and focus groups 
-Provides motivation and direction to the Self-study Team 
-Sets meeting schedule 
-Moderates discussions 

-Coordinates development of the drafts of the Self-study 
Report 

-Submits report as required 

-Acts as liaison with the Dean, Task Force/Program, 
Instructional Program Review Committee and Office of 
Institutional Effectiveness 


Full time faculty- 
in discipline 


1-3 


-Participates in discussions 

-Assists with collection and analysis of information 
-Assists with the development of the Self-study Report, 
including writing sections as needed 


Adjunct faculty- 
in discipline 


1-2 


-Participates in discussions 

-Assists with collection and analysis of information 
-Assists with the development of the Self-study Report 


Full time faculty- 
not in discipline 


1-2 


-Participates in discussions 

-Provides insight into relationship of service courses to the 
instructional area under review, including the collection and 
analysis of information 

-Assists with the development of the Self-study Report 


Students (current 
and/or former) 


1-2 


-Participates in discussions 

-Provides insight from student perspective 

-Assists with the development of the Self-study Report 


Community 
members/ external 
consultant 


1-2 


-Participates in discussions 

-Provides insight from the perspective of business, industry, 
or other college/university 

-Assists with the development of the Self-study Report 


Dean (ex officio) 


1 


-Participates in discussions, but not in writing of Self-study 
Report 
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External consultants may be included as part of the program review process. The 
Dean in consultation with the appropriate Associate Vice President and Taskforce 
Chair/Program Coordinator will make the determination. 



Program review process 

This timeline is to serve as a guide. Modifications may need to be made based on 
institutional constraints. 

Prior to starting review 

Instructional Program Review Committee members identified by instructional 
Executive Vice President 

Guidelines for Instructional Program Review developed 

Training for Instructional Program Review Committee members and Self-study 
Team developed 



Spring I - Summer I 

Programs to undergo review notified (January) 

Instructional Program Review Committee meets with Program 
Coordinators/Taskforce Chairs to review process 

Program/Task Force identifies Self-study Team membership and makes 
recommendations to the Dean. 

Dean reports membership of Self-study Team to Instructional Program Review 
Committee 

Surveys of students, faculty, employers, etc. conducted by Program/Task Force 
and/or Self-study Team 

Data on indicators provided by Office of Institutional Effectiveness 

Falll 

Self-study Team conducts self-study, with assistance from the Instructional 
Program Review Committee 

Self-study Team reviews draft Self-study report and recommendations with 

Program/Task Force and Dean 

Institutional Effectiveness Measures reviewed and revised 

Implications for strategic planning, operation planning, and budgeting determined 

Taskforce Chair/Program Coordinator proposes Action Plans to address 
recommendations 
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Self-study Report completed by December 1 5 

Self-study sent to Instructional Program Review Committee for review of 
completeness and quality 

Self-study sent to Dean for review of results 

Spring II 



Preliminary Self-study Report reviewed with Dean and Associate Vice President 
(January) 

Associate Vice President and instructional Executive Vice President review the 
Self-study Report 

If necessary, Self-study Team revises Self-study Report 

Associate Vice President incorporates recommendations into institutional 
planning and budgeting processes 

By April 15, Dean files final Self-study Report Associate Vice President and the 
instructional Executive Vice President 

Summary of Self-study Report prepared by Self-study Chair and Dean presented 
to Board of Trustees (Information Item) 

Spring III 

Progress report completed by Taskforce Chair/Program Coordinator as part of 
institutional effectiveness process 
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Program Review Schedule 



In consultation with the administration, area Deans are responsible for 
determining the order in which programs will be evaluated. During the first cycle, 
programs will be reviewed as indicated in the chart below. The Instructional Program 
Review Committee will consult with the Deans each fall prior to notifying programs. 



Dean Area 


1999-2000 


2000-2001 


2001-2002 


2002-2003 


Applied 


Photography 


ACR 


BCT 


Human Services 


Technologies, 


Commercial Art 


Criminal Justice 


Welding 


Technical 


\yf i il ti m pH i 


Printing 


Fire Protection 


Child Development 


Communications 




Commercial Music 


Automotive 






and Public 










Service 










Computer 


Quality Assurance 


EDG 


SMT 


SRV 


Studies and 


CIS/CSC 




CER 




Advanced 


ELN 








Technology 










Business 


OST 


Marketing 


Accounting 


Travel 


Studies 


FNM 


Fashion 


Legal 


Hospitality 




BMG 


Real Estate 




Culinary Arts 




MGT 








Health 


DMS 


OTA 


PHR 


VNG 


Sciences 


MLT 


SRG 


FIT (PFT) 


NSG 




DMR 


PTA 


EMS 










AHS 




Math and 


Mathematics (including 


Engineering 


Biology 


Astronomy 


Sciences 


Developmental) 


Physics 


Physical Science 


Health & Kinesiology 




Chemistry 




Geology 




Social and 


History 


Anthropology 


Psychology 


Government 


Behavioral 




Geography 


Sociology 




Siripnrp^ 




Economics 


Human Development 




UvlvllvvO 






Women’s Studies 




Arts and 


Developmental Reading 


Foreign Languages 


English 


Art 


Humanities 


Developmental Writing 


Radio-TV-Film 


Speech 


Dance 




English as a Second 




Journalism 


Drama 




Language 






Music 










Philosophy/ Religion 










Humanities 



Modifications to this schedule may be made in consultation with the instructional 
Executive Vice President. Where programs are closely related, one Self-study Team may 
be appointed to review multiple programs. This should be coordinated with the Dean and 
Associate Vice President. In these cases, the membership of the Self-study Team may be 
expanded and must include at least one full-time faculty member from each area under 
review. 
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Indicators of Effectiveness 



The basis of the Program Review Process is the analysis of information relating to 
a set of Core Indicators. These Core Indicators have been identified by the Program 
Review Work Group and reflect input from a variety of sources. They include indicators 
federal, state and regional agencies, including SACS, the LBB, and the THECB, as well 
as “best practices” identified in similar processes at other community colleges. 

As part of the Self-study process. Self-study Teams in consultation with the Dean 
and Instructional Program Review Committee will select the indicators on which their 
analysis will focus. The only exception to this is a set of required indicators that are 
defined by SACS, the LBB, the THECB, other agencies, and/or institutional priorities. 
These are noted in parenthesis in the tables below. 

Codes are as follows: 

SOURCE 

P = Provided in Documentation Notebook 
I = Instructional area must provide 

PRIORITY 
R = Required 
O = Optional 

A = All instructional areas 
W = Workforce 
T = Transfer 

The Core Indicators are organized into three sets— Need, Cost, and Program 
Effectiveness— defined as follows: 

Need: Does the program address a verifiable need of students, the community, or 

society? 

Cost: Within the context of the College’s mission, is the cost of the program 

justified? 

Program 

Effectiveness: Are the teaching, learning, course, program, student and student support 
outcomes of this program of the best possible quality? 



NEED 

Does the program address a verifiable need of students, the community, or society? 



Need 



Src 


Pri 


Core Indicators 


Type of data/information 


P 


R-A 


Enrollment trends (previous 5 years) 


Student enrollment in courses 


P 


R-A 




Number of sections offered 




R-W 




Number of declared majors (beginning when 
Degree Audit is implemented) 


P 


R-A 


Graduates (previous 3 years) 


Number of degrees conferred {THECB, LBB} 


P 


R-W 


Outcomes (previous 3 years) 


Percent of graduates who are employed with in 
one year of graduation {THECB} 


P 


R-A 




Number and Percent of graduates who 
Transfer with in one year of graduation 
{THECB, LBB} 


P 


R-W 




Percent of employed graduates who indicate 
employment is related to training (ACC 
Survey of Graduates) 


P/I 


O-A 




Analysis of courses transferred 


I 


O-W 


Labor market opportunities 


Labor market information 


P/I 






Employer input — Survey, focus groups 


I 


O-A 


Competition from other institutions 


Other options for students in the Service 
Delivery Area 



COST 

Within the context of the College’s mission, is the cost of the program justified? 



Cost 







Core Indicators 


Type of data/information 


I 


O-A 


Budget analysis (previous 2 years) 


Analysis of actual expenditure related to 
program compared to budget 


P 


R-A 


Comparative program costs (previous 2 
years) 


Ratio of non-faculty program costs to contact 
hours generated 


P 


R-A 


Program costs (previous 2 years) 


Cost per contact hour consistent with type of 
instruction 
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PROGRAM EFFECTIVENESS 

Are the teaching, learning, course, program, student and student support outcomes of this 
program of the best possible quality? 



Student Achievement 







Core Indicators 


Type of data/information 


p 


R-A 


Course completion rates 


Withdrawal rates for courses {SACS, THECB, 
LBB} 

D-F-W rates within college norms 


p 


R-A 


Program (degree/certificate) 
completion 


Number of students completing 
degree/certificate requirements {THECB, 
LBB} 


I 


O-A 


Capstone assessment 


Number/Percent of students completing 
capstone experience {THECB} 


P/I 


O-W 


Licensure pass rates 


Percent of graduates passing licensure exams 
{THECB, LBB} 


p 


O-A 


Student transfer rates 


Number of students transferring {THECB, 
LBB} 


I 


O-A 


Comparison with external norms 


Pass rates on nationally normed tests {SACS} 



Student Retention 







Core Indicators 


Type of data/information 


P 


R-A 


Course retention rates within college 


Withdrawal rates for courses {LBB} 






norms 




P 






D-F grade rates within college norms 


P 


R-W 


Completion rates of declared major 
with intent to complete a 
degree/certificate 


Number of declared majors intending to 
complete a program who complete 
degree/certificate requirements with in 6 years 
{THECB, IPEDS} 



Student Outcomes 







Core Indicators 


Type of data/information 


P 


R-W 


Employment of graduates within 
THECB standards 


Jobs placement rates from THECB follow-up 
{THECB, LBB} 


P/I 






Employment related to program from ACC 
survey of graduates 


P 


R-T 


Transfer of graduates/course 
completers 


Number of course takers transferring with in 
one year 




O-T 




Number of course takers who indicate intent to 
transfer who do transfer with in one year of 
leaving ACC (When intent data are available) 
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P/I 


R-T 




GP A of transfer students at transfer institution 


p 


R-W 


Meet THECB standards for number of 
Graduates 


Number of program graduates in 3 years 
{THECB} 


I 


O-A 


Determination of proficiency levels 


Departmental student assessment (e.g., 
departmental final exams, exit tests, 
standardized testing, etc.) 



Access 







Core Indicators 


Type of data/information 


P 


R-A 


Course availability 


Analysis of when and where courses are 
offered by campus, time of day, mode of 
delivery 


P 


R-A 


Extent to which courses meet student 
demand 


Number of sections taught by location 


P 






Number of sections closed/cancelled by course 


P 






High demand course analysis 


I 


O-A 


Marketing of program/courses 


Program brochures 


I 






Description of program and institutional 
outreach activities 


P 


O-A 


Access by various types of students 


Demographics of students taking courses 


P 


R-W 




Demographics of students with declared 
majors and intent to complete program 
{THECB} 


I 


O-A 


Accessibility of course content 


Student course evaluation questions related to 
course content 


P 


R-W 


Length of time it takes to complete 


Average number of semesters it takes for . 






program 


students to gain degree/credential 



Curriculum and Instruction 







Core Indicators 


Type of data/information 


I 


R-W 


Currency of program/course content 


Date of most recent program revision 
{THECB} 


I 


R-A 




Course texts up-to-date {THECB} 


I 


R-A 


Learning outcomes defined 


Learning outcomes defined for courses and 
programs {THECB} 


I 


R-A 


Course syllabi 


All courses have up-to-date syllabi on file 
{SACS, THECB} 



ERIC 
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I 


0-A 




Number of courses with departmental syllabi 


I 


R-A 


Faculty credentials 


All faculty meet SACS standards {SACS, 
THECB} 


I 


0-A 


Faculty satisfaction 


Rate of faculty turnover 


I 


0-A 




Focus groups/surveys 


p 

p 


R-A 


Number of faculty adequate to teach 
courses 


Percent of sections taught by full-time faculty 

Percent of contact hours taught by full time 
faculty {LBB} 


I 


0-A 


Faculty professional development 


Number involved in formal professional 
development activities 


I 






Types of discipline-related professional 
development activities offered 


I 


R-A 


Teaching effectiveness 


Student evaluations of instruction within 
acceptable range 


p 


R-A 




Extent to which alternative modes of 
instruction are incorporated into classes 


I 


0-A 




Extent to which focus of instruction is on 
problem solving, active learning, and work- 
based elements 


I 


R-W 


External advisory committee input 


Committee meets regularly to discuss 
curriculum issues 


I 


R-W 




Most recent review of curriculum by external 
advisory committee {THECB} 


I 


0-A 


Equipment assessment 


Assess area use of instructional equipment 
{THECB} 


I 


0-A 




Extent to which courses rely on specific 
equipment 


I 


R-A 


Technology assessment 


Extent to which technology impacts mode of 
instruction 


p 


R-A 




Number of courses/sections taught via distance 
learning 


I 


0-A 




Number of courses/sections that incorporate 
multi-media 


I 


0-A 




Number of courses/sections taught using 
alternative instructional methodology 
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I 


R-A 


Assessment of instructional resources 


Evaluation of the use of instructional resources 
including those in the LRS 


I 


O-A 


Support courses meet the needs of 
other disciplines and programs 


Extent to which courses are required by other 
courses and programs 


I 


O-A 




Pre-requisite tracking — how students perform 
in subsequent courses 


I 


O-A 




Input from other programs and disciplines 
regarding learning outcomes and student 
performance 


p 


O-A 


Appropriateness of course pre- 
requisites 


Analysis of course completion data 


I 


O-A 




Analysis of course competencies. Work Keys, 
DACUM, ACAP, etc. 


I 


O-A 




Analysis of scope and sequence of curriculum 


I 


O-A 


Course effectiveness 


Departmental course assessment (e.g., exit 
tests, surveys, standardized testing, syllabus 
review, etc.) 


I 


R-A 


Catalog content up to date 


Analysis of course and program listings in 
ACC catalog 


I 


R-W 


SCANS incorporated into program 


Analysis of documentation of the curriculum 
{THECB} 



Diversity 







Core Indicators 


Type of data/information 


P 


R-A 


Student diversity 


Demographics of students taking courses 




R-W 




Demographics of students indicating intent to 
complete program (Available when intent data 
are accessible) 


I 


O-A 


Faculty diversity 


Demographics of full time and adjunct faculty 


I 


O-A 


Outreach activities 


Activities to reach under-represented 
populations 


P 


R-W 


Diversity of graduates 


Demographics of graduates (LBB, THECB} 



Student Satisfaction 







Core Indicators 


Type of data/information 


I 


R-A 


Satisfaction of students with courses 


Student course evaluations 


P 


R-W 


Satisfaction of graduates with program 


ACC Graduate Survey data 


I 


O-A 




Area surveys 
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Employer Satisfaction 







Core Indicators 


Type of data/information 


p 


R-W 


Employment of graduates 


Percent of graduates employed with in one 
year {THECB} 


p 


R-W 


Employer satisfied with program 
completers’ proficiency 


Percent of employers indicating satisfaction 
with graduates {THECB} 


I 


O-W 




Focus group or internal survey of employers 


I 


O-W 


Employers satisfied with number of 
program completers 


Number of employers indicating need for more 
graduates 



Transfer Institution Satisfaction 







Core Indicators 


Type of data/information 


I 


R-T 


Satisfaction of transfer institutions 


Survey/focus group of transfer institutions 
{THECB} 


P 


O-A 




Data from transfer institutions 


P 


O-A 


Level of student transfer 


Number of students transferring successfully 
{THECB} 


I 


R-T 


Transfer articulation agreements 


Number of agreements with universities and 
colleges {THECB} 


I 


R-T 


Extent to which courses transfer 


Number of courses which transfer 


I 


R-T 




Number of students who contact college about 
problems with course transfer 



Institutional Effectiveness 







Core Indicators 


Type of data/information 


P 


R-A 


Institutional Effectiveness Measures 


Institutional effectiveness measures and results 
for past 3 years {SACS} 


I 


R-A 




Extent to which measures reflect required 
learning outcomes 



Instructional Program Review Process 

The Self-study begins with an analysis of the information listed above. The Self- 
study Team then considers the following: 

Review of Mission/Vision 

The Mission of the instructional area will be reviewed to ensure that it continues 
to reflect the need for the program/discipline. 
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Identification of Strength, Weaknesses, Opportunities and Threats 

As part of the Self-study, the program will conduct a SWOT to determine the 
strategic priorities that need to be addressed. The data collected in the Need, 

Cost, and Program Effectiveness sections form the basis for the SWOT. 

Integration of Baldrige Criteria (optional) 

Programs interested in using quality-based criteria or pursing one of the quality 
awards may do so as part of the Program Review process. Specific information is 
available through the Office of Institutional Effectiveness. 

This process is coordinated by the Greater Austin Quality Council (GAQC), 
which is a component of the Greater Austin Chamber of Commerce. The GAQC 
program offers public recognition and private assessments, via the Greater Austin 
Quality Awards, to organizations in the Greater Austin Region for quality and 
performance excellence. 

Using the Malcolm Baldrige National Quality Award criteria, private written 
assessments (Feedback Reports), which describe an organization’s strengths and 
areas for improvement as measured and with performance excellence criteria are 
the major features of the program. 

The core values and concepts of the Malcolm Baldrige National Quality are 
embodied in seven categories: 

1 . Leadership 

2. Strategic Planning 

3. Customer and Market Focus 

4. Information and Analysis 

5. Human Resource Focus 

6. Process Management 

7. Business Results 

Recommendations 

The Self-study Team should identify the major issues that need to be addressed as 
a result of information gathered during the self-study process. It should then 
identify what will be done to address the issues identified. In most cases, 5 to 10 
recommendations will be made. 

Action Plans 

As part of the Self-study, the Taskforce/Program must develop Action Plans for 
each Recommendation. Action Plans are proposals for change and may serve as 
guidelines for future planning activities. 
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Institutional Effectiveness Measures 



In addition, at least three Institutional Effectiveness Measures need to be 
identified and included as part of the Self-study Report. Assistance on developing 
these can be provided by the Office of Institutional Effectiveness. 

Implications for the Budget Process 

The Taskforce Chair/Program Coordinator should develop an outline of 
operational planning priorities for next budget cycle. This should be shared with 
the Dean and Associate Vice President during the budget process. 
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